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DETAILED ACTION 

1 . This office action is in response to correspondence filed October 31 , 2007 in 
reference to application 10/812,000. Claims 1-20 are pending in the application and 
have been examined. 

Response to Amendment 

2. The amendments to the claims filed October 31 , 2007 have been accepted and 
considered in this office action. Claims 1, 2, 4, 8, 11, 13, 14, 16-18 and 20 have been 
amended. 

Response to Arguments 

3. Applicant's arguments with respect to claims 1 - 20 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 
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1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

3. Claims 1,13, and 20 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nagata (A Stochastic Japanese Morphological Analyzer Using a Forward-DP 
Backward-A N-Best Search Algorithm) in view of Aires (Combining Multiple Classifiers 
to Improve Part of Speech Tagging.) 

4. Claim 1,13, and 20 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Nagata (A Stochastic Japanese Morphological Analyzer Using a Forward-DP 
Backward-A N-Best Search Algorithm). 

5. Consider claim 1 , Nagata teaches a morphological analyzer comprising 
(Japanese morphological analyzer, abstract): 

a hypothesis generator for applying a prescribed method of morphological 
analysis to a text and generating one or more hypotheses as candidate results of the 
morphological analysis, each hypothesis being a word string with part-of-speech tags, 
the part-of-speech tags including form information for parts of speech having forms (We 
propose a novel search strategy for getting the N best morphological analysis 
hypotheses for the input sentence; page 201 , column 1 , line 35. We used tile simple tri- 
POS model as the tagging model for Japanese; page 201 , column 1 , line 30.); 
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a model storage facility storing information for a plurality of part-of-speech n- 
gram models, at least one of the part-of-speech n-gram models including information 
about the forms of the parts of speech (We used tile simple tri-POS model as the 
tagging model for Japanese; page 201, column 1, line 30. Tagging model also 
described in detail in section 2. It is inherent that the model must be stored in a memory 
in order to enable the analyzer.); 

a probability calculator for finding a probability that each said hypothesis will 
appear in a large corpus of text by using a weighted combination of the information for 
the part-of-speech n-gram models stored in the model storage facility (equations 3, 4 
and 5 described in section 201 estimates the probabilities of the tagging with relation to 
the relative frequencies of the corresponding events.); and 

a solution finder for finding a solution among said hypotheses, based on the 
probabilities generated by the probability calculator (Once word hypotheses for 
unknown words are generated, the proposed N-best algorithm will find tile most likely 
word segmentation and part of speech assignment taking into account the entire 
sentence; page 204, column 1, line 22.) 

Nagata does not specifically teach at least two of the part-of-speech n-gram 
models being based on mutually different types of morphological information. 

In the same field of POS tagging, Aires teaches at least two of the part-of-speech 
n-gram models being based on mutually different types of morphological information 
(four different classifiers were used, including a Unigram and an n-gram tagger; page 2 
paragraph 1 . The unigram and n-gram are two different n-gram models that are based 
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on different morphological information as the unigram considers just one 
word/morpheme at a time and the n-gram considers the context of n 
words/morphemes,). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine the morphological analyzer of Nagata with the multiple part 
of speech models as taught by Aires in order to obtain a higher accuracy in tagging, 
(abstract Aires). 

6. Consider claim 13, A method of morphological analysis comprising (Japanese 
morphological analyzer, abstract): 

applying a prescribed method of morphological analysis to a text and generating 
one or more hypotheses as candidate results of the morphological analysis, each 
hypothesis being a word string with part-of-speech tags, the part-of-speech tags 
including form information for parts of speech having forms (We propose a novel search 
strategy for getting the N best morphological analysis hypotheses for the input 
sentence; page 201 , column 1 , line 35. We used tile simple tri-POS model as the 
tagging model for Japanese; page 201 , column 1 , line 30 ); 

calculating probabilities that each said hypothesis will appear in a large corpus of 
text by using a weighted combination of a plurality of part-of-speech n-gram models, at 
least one of the part-of-speech n-gram models including information about forms of 
parts of speech (equations 3, 4 and 5 described in section 201 estimates the 
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probabilities of the tagging with relation to the relative frequencies of the corresponding 
events.); and 

finding a solution among said hypotheses, based on said probabilities (Once 
word hypotheses for unknown words are generated, the proposed N-best algorithm will 
find tile most likely word segmentation and part of speech assignment taking into 
account the entire sentence; page 204, column 1, line 22.). 

Nagata does not specifically teach at least two of the part-of-speech n-gram 
models being based on mutually different types of morphological information. 

In the same field of POS tagging, Aires teaches at least two of the part-of-speech 
n-gram models being based on mutually different types of morphological information 
(four different classifiers were used, including a Unigram and an n-gram tagger; page 2 
paragraph 1 . The unigram and n-gram are two different n-gram models that are based 
on different morphological information as the unigram considers just one 
word/morpheme at a time and the n-gram considers the context of n 
words/morphemes.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine the morphological analyzer of Nagata with the multiple part 
of speech models as taught by Aires in order to obtain a higher accuracy in tagging, 
(abstract Aires). 

7. Consider claim 20, Nagata teaches a machine-readable tangible medium storing 
a program comprising instructions that can be executed by a computing device to carry 
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out morphological analysis by the method of claim 13 (Nagata teaches a morphological 
analyzer; abstract. In order to operate this analyzer it is inherent that there is in fact ca 
storage medium containing the instructions in order to operate it. ). 

8. Claims 2, 3, 14, and 15 are rejected under 35 U.S. C. 103(a) as being 
unpatentable over Nagata and Aires as applied to claims 1 and 13 above and further in 
view of Haruno et al. (Mistake-Driven Mixture of Hierarchical Tag Context Trees). 

9. Consider claim 2, Nagata and Aires teaches the morphological analyzer of claim 
1 , but does not specifically teach wherein said at least one of the part-of-speech n-gram 
models including information about forms of parts of speech is a hierarchical part-of- 
speech n-gram model that treats parts of speech and their forms at different hierarchical 
levels. 

In the same field of tagging, Haruno teaches a part-of-speech n-gram model 
including information about forms of parts of speech is a hierarchical part-of-speech n- 
gram model that treats parts of speech and their forms at different hierarchical levels 
(the tags are representing in a hierarchical tag context tree, abstract. Figure 1 shows 
parts of speech and their forms on different hierarchical levels.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine the hierarchical model of Haruno with the morphological 
system of Nagata and Aires in order to use a method of tagging that outperforms both 
hand-crafted and conventional statistical methods, abstract Aires. 
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Consider claim 3, Haruno suggests the morphological analyzer of claim 2, wherein the 
hierarchical part-of-speech n-gram model calculates a product of a conditional 
probability P(wi I ti) of occurrence of a word wi given its part of speech ti, a conditional 
probability P(tiform I tipos) of occurrence of the part of speech tipos of said word wi in a 
form tiform shown by said word wi, and a conditional probability P(tiPos I ti-N+l...ti-l) of 
occurrence of the part of speech tipos of said word wi following a part-of-speech tag 
string ti-N+l...ti-i indicating parts of speech of N - 1 preceding words, where N is a 
positive integer (Page 231, equation 1 shows multiplying P(Wi|ti) with P(ti|t1,i-1). It 
would be obvious to one of ordinary skill in the art when considering separate forms of a 
word to add the conditional probability of P(tiform|tipos) to the product in order to 
provide The probability of the further subdivisions of the part of speeches. 

10. Consider claim 14, Nagata teaches method of claim 13, but does not specifically 
teach wherein said at least one of the part-of-speech n-gram models including 
information about forms of parts of speech is a hierarchical part-of-speech n-gram 
model. 

In the same field of tagging, Haruno teaches a part-of-speech n-gram model 
including information about forms of parts of speech is a hierarchical part-of-speech n- 
gram model that treats parts of speech and their forms at different hierarchical levels 
(the tags are representing in a hierarchical tag context tree, abstract. Figure 1 shows 
parts of speech and their forms on different hierarchical levels.). 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine the hierarchical model of Haruno with the morphological 
system of Nagata and Aires in order to use a method of tagging that outperforms both 
hand-crafted and conventional statistical methods, abstract Aires. 

Consider claim 15, Nagata teaches the method of claim 14, wherein the hierarchical 
part-of-speech n-gram model calculates a product of a conditional probability P(wi I ti) of 
occurrence of a word wi given its part of speech ti, a conditional probability P(tiform I 
tipos) of occurrence of the part of speech tipos of said word wi in a form tiform shown by 
said word wi, and a conditional probability P(tiPos I ti-N+l...ti-l) of occurrence of the part 
of speech tipos of said word wi following a part-of-speech tag string ti-N+l...ti-i indicating 
parts of speech of N - 1 preceding words, where N is a positive integer (Page 231 , 
equation 1 shows multiplying P(Wi|ti) with P(ti|t1,i-1). It would be obvious to one of 
ordinary skill in the art when considering separate forms of a word to add the conditional 
probability of P(tiform)tipos) to the product in order to provide The probability of the 
further subdivisions of the part of speeches. 

11. Claims 4-7, and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nagata in view of Aires as applied to claims 1 and 13 above and further in view of 
Lee et al (Lexicalized HMM's for Part-of-Speech Tagging). 
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12. Consider claim 4, Nagata in view of Aires teaches the morphological analyzer of 
claim 1 , but does not specifically teach wherein at least one of the part-of-speech n- 
gram models is a lexicalized part-of-speech n-gram model in which all words are 
lexicalized. 

In the same field of part of speech tagging, lee teaches at least one of the part- 
of-speech n-gram models is a lexicalized part-of-speech n-gram model (This paper 
discusses uniformly lexicalizing HMM models, that is lexicalizing all HMM models, 
instead of doing so selectively; abstract and introduction paragraphs 2 and 3). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to used lexicalized HMMs as taught by Lee with the analyzer of Nagata 
and Aires in order to improve tagging accuracy; Lee abstract. 

13. Consider claim 5, Lee suggests the morphological analyzer of claim 4, wherein 
the lexicalized part-of-speech n-gram model calculates a product of a conditional 
probability P(wi I ti) of occurrence of a word wi given its part of speech ti and a 
conditional probability P (ti I Wi-N+1 ti-N+l ... Wi-1 ti-1) of occurrence of the part of 
speech ti of said word wi following N - 1 words wi-N+1 wi-1 having respective parts of 
speech ti-N+1 ... ti-1 , where N is a positive integer (Equation 4 and Equation 5 can be 
manipulated to create the production of P(wi I ti) and P (ti I Wi-N+1 ti-N+l ... Wi-1 ti-1 . 
As equation 5 is a derivative of equation 4 for lexicalized HMM's). ). 
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14. Consider claim 6, Lee teaches the morphological analyzer of claim 4, wherein the 
lexicalized part-of-speech n-gram model calculates a conditional probability P(witi I ti- 
N+1 ...ti-1 ) of occurrence of a word wi having a part of speech ti following a string of N - 

1 parts of speech ti-N+1 ..-ti-1 , where N is a positive integer (The probability of the 
current tag on the current word depends of the previous k tags; section 3.1 Therefore 
the equation of claim 6 is in obvious computation to make.). 

1 5. Consider claim 7, Lee teaches the morphological analyzer of claim 4, wherein the 
lexicalized part-of-speech n-gram model calculates a conditional probability P(witi I wi- 
N+1 ti-N+l. . .Wi-1 ti-1) of occurrence of a word wi having a part of speech ti following a 
string of N - 1 words wi-+1...wi-i having respective parts of speech ti-N+1... ti-1, where N 
is a positive integer (The probability of the current tag on the current word depends of 
the previous k tags and the K previous words; section 3.1 Therefore the equation of 
claim 7 is in obvious computation to make.) 

16. Consider claim 16, Nagata and Aires teaches the method of claim 13, but does 
not specifically teach wherein at least one of the part-of-speech n-gram models is a 
lexicalized part-of-speech n-gram model in which all words are lexicalized. 

In the same field of part of speech tagging, Lee teaches at least one of the part- 
of-speech n-gram models is a lexicalized part-of-speech n-gram model (This paper 
discusses uniformly lexicalizing HMM models, that is lexicalizing all HMM models, 
instead of doing so selectively; abstract and introduction paragraphs 2 and 3). 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to used lexicalized HMMs as taught by Lee with the analyzer of Nagata 
in order to improve tagging accuracy; Lee abstract. 

17. Claims 8, 11, 17, and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nagata in view of Aires, as applied to claims 1 and 13 above, and 
further in view of Schone et al (Language-independent Induction of Part of Speech 
Class Labels Using only Language Universals). 

1 8. Consider claim 8, Nagata and Aires teaches the morphological analyzer of claim 
1 , but does not specifically teach wherein at least one of the part-of-speech n-gram 
models stored in the model storage facility is a class part-of-speech n-gram model 
employing classes obtained by clustering. 

In the same field of n-gram language modeling, Schone teaches at least one of 
the part-of-speech n-gram models stored in the model storage facility is a class part-of- 
speech n-gram model employing classes obtained by clustering (This algorithm derives 
clusters and part of speech tag; page 1 column 2 paragraph 2. ). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use class n-gram models as taught by Schone for parsing as taught 
by Nagata and Aires in order to provide a more accurate part of speech analysis. 
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1 9. Consider claim 1 1 , Schone teaches the morphological analyzer of claim 8, 
wherein the class part-of-speech n-gram model is trained from both a part-of-speech 
tagged corpus and a part-of-speech untagged corpus, clustering parameters obtained 
from the part-of-speech untagged corpus being used to cluster morphemes in the part- 
of-speech tagged corpus (Section 2 describes the clustering, and the that the only 
previous knowledge is previously known grammar rules. No tagged corpus is used. ). 

20. Consider claim 17, Nagata and Aires teaches the method of claim 13, but does 
not specifically teach wherein at least one of the part-of-speech n-gram models stored 
in the model storage facility is a class part-of-speech n-gram model employing classes 
obtained by clustering. 

In the same field of n-gram language modeling, Schone teaches at least one of 
the part-of-speech n-gram models stored in the model storage facility is a class part-of- 
speech n-gram model employing classes obtained by clustering (This algorithm derives 
clusters and part of speech tag; page 1 column 2 paragraph 2. ). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use class n-gram models as taught by Schone for parsing as taught 
by Nagata and Aires in order to provide a more accurate part of speech analysis. 

21 . Consider claim 1 8, Schone teaches method of claim 1 7, teaches wherein the 
class part-of-speech n-gram model is trained from both a part-of-speech tagged corpus 
and a part-of-speech untagged corpus (Section 2 describes the clustering, and the that 
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the only previous knowledge is previously known grammar rules. No tagged corpus is 
used. ). 

22. Claim 9-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nagata and Aires in view of Schone as applied to claim 8 above, and further in view of 
Lee et al. 

23. Consider claim 9, Nagata, Aires and Schone teaches the morphological analyzer 
of claim 8, but does not teach specifically wherein the class part-of-speech n-gram 
model calculates a product of a conditional probability P(wi I ti) of occurrence of a word 
wi given its part of speech ti and a conditional probability P (ti I Ci-N+1 ti-N+1 ... Ci-1 ti- 
1) of occurrence of said part of speech ti following a string of N - 1 words assigned to 
respective classes Ci-N+1. ..Ci-1 with respective parts of speech ti-N+1. ..ti-1, where N is 
a positive integer. 

In the same field of tagging, Lee teaches lexical part-of-speech n-gram model 
calculates a product of a conditional probability P(wi I ti) of occurrence of a word wi 
given its part of speech ti and a conditional probability P (ti I Ci-N+1 ti-N+1 ... Ci-1 ti-1) of 
occurrence of said part of speech ti following a string of N - 1 words assigned to 
respective classes Ci-N+1. ..Ci-1 with respective parts of speech ti-N+1... ti-1, where N is 
a positive integer (The probability of the current tag on the current word depends of the 
previous k tags and the K previous words; section 3.1 This can be combined with the 
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induction probabilities of Schone described in section 3. Therefore the equation of claim 
9 is in obvious computation to make. This can be combined) 

Therefore it would have been obvious to apply the same rules of probability 
equations for tagging as taught by Lee to the class tagger of Nagata and Schone and 
Aires in order to provide a more accurate tagging, taking fully into consideration the 
class of which the words belong. 

24. Consider claim 10, Nagata, Aires and Schone and Aires teaches the 
morphological analyzer of claim 8, but does not specifically teach wherein the class 
part-of-speech n-gram model calculates a product of a conditional probability P(witi I Ci- 
N+1 ti-N+1...Ci-1ti-1) of occurrence of a word wi having a part of speech ti following a 
string of N - 1 words in respective classes Ci-N+I...Ci-l With respective parts of speech 
ti-N+1...ti-1, where N is a positive integer. 

In the same field of tagging, Lee teaches wherein the lexical part-of-speech n- 
gram model calculates a product of a conditional probability P(witi I Ci-N+1 ti-N+1...Ci- 
1 ti-1 ) of occurrence of a word wi having a part of speech ti following a string of N - 1 
words in respective words Ci-N+I...Ci-l With respective parts of speech ti-N+1...ti-1, 
where N is a positive integer (The probability of the current tag on the current word 
depends of the previous k tags and the K previous words; section 3.1 This can be 
combined with the induction probabilities of Schone described in section 3. Therefore 
the equation of claim 10 is in obvious computation to make. This can be combined) 



Application/Control Number: Page 16 

10/812,000 

Art Unit: 2626 

Therefore it would have been obvious to apply the same rules of probability 
equations for tagging as taught by Lee to the class tagger of Nagata and Schone and 
Aires in order to provide a more accurate tagging, taking fully into consideration the 
class of which the words belong. 

25. Claims 12 and 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nagata and Aires as applied to claims 1 and 13 above, and further in view of Siu 
(Variable N-Grams and Extensions for Conversational Speech Modeling). 

26. Consider claim 12, Nagata and Aires teaches the morphological analyzer of 
claim 1, but does not specifically teach further comprising a weight calculation unit using 
a leave-one-out method to calculate weights of the part-of-speech n-gram models. 

In the same field of part of speech tagging, Siu teaches a weight calculation unit 
using a leave-one-out method to calculate weights of the part-of-speech n-gram models 
(To evaluate the effect of node pruning and merging on unseen data, we use leave-one- 
out (LOO) likelihood to estimate the distributions used in the distance measures; column 
69, column 2, section B.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the leave one out method as taught by Siu with the analyzer of 
Nagata and Aires in order to provide a more robust method of training. 
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27. Consider claim 1 9, Nagata and Aires teaches the method of claim 1 3, but does 
not specifically teach further comprising a weight calculation unit using a leave-one-out 
method to calculate weights of the part-of-speech n-gram models. 

In the same field of part of speech tagging, Siu teaches a weight calculation unit 
using a leave-one-out method to calculate weights of the part-of-speech n-gram models 
(To evaluate the effect of node pruning and merging on unseen data, we use leave-one- 
out (LOO) likelihood to estimate the distributions used in the distance measures; column 
69, column 2, section B.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the leave one out method as taught by Siu with the analyzer of 
Nagata and Aires in order to provide a more robust method of training. 

Conclusion 

28. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Douglas C. Godbold whose telephone number is (571 ) 
270-1451 . The examiner can normally be reached on Monday-Thursday 7:00am- 
4:30pm Friday 7:00am-3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached on (571) 272-7603. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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